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We present the constructive design of finite order equalizer filters for 
data transmission systems employing decision feedback equalization. 
Both transmitter design with power constraints and receiver design with 
ambient noise considerations are treated. Expressions for the filter tap 
settings which maximize a signal-to-noise ratio are found for both baseband 
pulse amplitude modulation and quadrature amplitude modulation 
(QAM) systems. Design examples are given in a passband equivalent 
(of QAM) formulation for an average toll telephone connection. Neglecting 
the possibility of error propagation, these examples demonstrate that 
decision feedback equalization requires fewer taps for acceptable system 
performance as compared to linear equalization. The problem of post- 
cursor size in a decision feedback equalized response is treated and shown 
to diminish in importance when a hybrid equalization procedure is 
imposed on the linear tap adjustment. The price one pays for allowing the 
linear filter taps to reduce the postcursor sizes in this hybrid equalizer is a 
lower signal-to-noise ratio. 

I. INTRODUCTION 

The advantage of using a nonlinear device, referred to as a decision 
feedback equalizer, to cancel the tails of pulses whose amplitudes have 
already been estimated in a PAM system has long been recognized. 
Figure 1 depicts the typical system in which the decision feedback 
mechanism has always been envisioned to perform this task. Namely, 
by making decisions on a symbol-by-symbol basis and by knowing 
the channel response precisely, a data system would be designed so 
that postcursor (tails of preceding pulses) ISI could be eliminated 
without the ambient noise penalty that a linear filter or equalizer 
imposes. The tacit assumption being made in any decision feedback 
implementation is that the signal-to-noise ratio is high without 

503 



N(f) NOISE 



A(z~ 



HgH 



H(f) 




+ 



LOW- 
PASS 



-®- 



Biz" 



J* 



TRANSMITTER 
FILTER 



CHANNEL 



FILTER 



RECEIVER DECISION 

FILTER FEEDBACK 

PROCESSOR 



Fig. 1 — Transmitter and receiver filter design. 

equalization, and correct decisions are already being made with high 
probability. 

In this paper we consider the design of finite order nonrecursive 
transmitting and receiving filters which counteract the two remaining 
sources of noise, the precursor tails which are the interfering samples 
of pulses whose amplitudes have not been decided upon, and ambient 
noise, everpresent in a communication system. In addition we seek 
the variation of the signal-to-noise ratio at the moment of decision 
when the sampling time is varied. Along with this variation of the 
criterion of system performance, we are also interested at each sampling 
time in the amount of postcursor ISI noise which the decision feedback 
mechanism is being asked to eliminate. This aspect of our investigation 
yields insight into the feasibility of decision feedback system implemen- 
tation. It is, of course, possible to design the transmitting and receiving 
filters to achieve "hybrid" equalization between simple linear equali- 
zation and decision feedback. That is, some of the linear filter's 
degrees of freedom will be used to combat some postcursor ISI, 
although decision feedback is being used. The idea is to reduce the 
possibility that large postcursor tails will be produced by a linear 
filter whose sole job would otherwise be to reduce precursor ISI. 

The system model we choose to work with is a sampled data or 
discrete one. In addition, the channel and the system's transmitting 
and receiving filters are assumed to be of finite nonrecursive type. 
Examples are discussed in a later section which involve voice-grade 
toll telephone channel spectra. These spectra have been reduced to a 
specified Nyquist equivalent bandwidth, both for baseband and 
passband applications. The timing involved in going from continuous 
waveforms to sampled data for these examples has been chosen to 
maximize a signal-to-noise criterion before any filtering is done at the 
receiver. Also, in the demodulation process for QAM, the carrier 
phase angle, if fixed, can be absorbed by the receiver's passband filter 
taps. (For more detail on the system model we use in the following 
sections, see Appendixes A and B.) 
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This paper follows two previous documents 1 ' 2 that have dealt with 
asymptotic performance results concerning decision feedback equaliza- 
tion. In these previous works, the filters assumed in the decision 
feedback equalization scheme were of infinite length. In contrast, we 
focus our attention here on designing filters of finite, implementable 
length and study a channel which is modeled from transmission data 
taken from the 1969-70 Toll Connection Survey of the Bell System. 

II. TRANSMITTER AND RECEIVER FILTER DESIGN (BASEBAND) 

We begin by referring to Fig. 1 and denoting the channel response 1 
by {hn}™. We are seeking nonrecursive filter tap weights {a„}o and 
{b„}o, N « M at the transmitter and receiver, respectively. We note 
the total response through the system is then 

{rn] *+2*+i = [ciMMSMM?. (i) 

where * denotes sequence convolution. 

If we decide to sample at time r and cancel* r^, k > t through 
decision feedback, then we can define a signal-to-noise ratio 

p(^r,a t b)A ||a ^ Lr2 (2) 

representing the sampled signal in the numerator and two noise terms 
in the denominator. The first noise term consists of the ambient noise 
which is modified by the receiving filter. [We write b for (6o, &i, • ■ ■ , b n ) 
in E N+1 Euclidean space with (a, b) as the usual inner product and 
||b|| 2 = (b, b) the usual norm.] We have assumed that the noise 
samples are independent and of generalized variance 5 a 2 and that the 
input binary stream of symbols is independently and fairly signed and 
of unit magnitude. The second denominator term is a measure of the 
precursor 181. 

2.1 Filter design by integral adjustment 

If we assume that the transmitter filter is to be optimized indepen- 
dently from the receiver filter we are then concerned with the 



f Appendix A explains our use of the sampled response \h„\. We suppress the 
constant multiplier l/T which converts the z~ l coefficients to time samples (where 
T is the time between samples). 

' We choose to cancel all postcursors. In practice, only a few are cancelled and 
others then become part of ISI term in (2). 

5 By generalized variance we imply that a constant multiplies the true noise sample 
variance. This constant takes into account the sampling speed at which we are 
measuring the signal-to-noise ratio. 
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response : 

{gn\¥ +N = l«»)M^io tf - (3) 

Define a = (a , a h • • •, a N ) and h fc = (h k , h k - X , ••-, /i*_*). We seek 
the maximum of 

><*■*•*> = ?+%fc* (4) 

fc<r 

subject to the constraint that ||a|| 2 = pA That is, we place an average 
power constraint on the transmitter. Hence, by constructing the 
quadratic form induced 3 by the sum of bilinear forms (h fc , a) 2 , we 
reshape p (N, t, a) into 

'(^ »> %->(», ^+(a, Q «y (5) 

where J is the (N + 1) X (N + 1) identity matrix and Q is the 
(AT + 1) X (N + 1) positive semidefinite matrix (£*<r h k -ih k -j) 
£i, j £ N with /&_* = 0, ! > 0. By use of the Cauchy-Schwartz 
inequality we find readily that the maximum of p(iV,r,a) is achieved at 

and the maximum is precisely 

max P (N, t, a) = P (N, r, a*) = (h T , (p-V-7 + Q)~ l h T ). (7) 

ll«ll*-M* 

We note that the sequence (h , fa, • • • , K) is mapped by the vector a* 
into a sequence (go, g\, • • • , gl) which the receiver is now expected to 
process in forming the following signal-to-noise ratio : 

where 

g* = (S't, fft-i, •• •> S't-jv)- 

Since p(iV, r, a*, b) is invariant to any scaling of b, we choose to 
maximize the former with respect to ||b|| = 1. By the same argument 
which led us to (6) and (7), we find 

b ~ \\l0*I + Rj-tfrW K 
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and 

P (N, r, a*, b*) = ||[yj + 1 fi] - lg; || (g' - VI + Brtr*), (10) 

where # is the (iV + 1) X (N + 1) matrix (£ k<r gl- t gl-j), ^ i, 
j ^ AT. Hence, p(N, r, a*, b*) in (10) represents the maximum signal- 
to-noise ratio achievable through the integral or independent adjust- 
ment of transmitter and receiver filters for a decision feedback system 
committed to sampling at time r and constrained to use nonrecursive 
linear filters of length N + 1. 

The difference between linear equalization and decision feedback 
can be seen readily by observing the denominator terms of the follow- 
ing signal-to-noise ratio: 

p{N ' T > a ' b) = „'||b||« + E (fob)' + E (g*, b)«- (U) 

For decision feedback systems, the last term in the denominator 
does not enter the picture because it is assumed it will be eliminated 
without noise penalty. However, in linear equalization, the filter b 
is expected not only to combat precursor ISI but postcursor ISI as 
well, with as little compromise to ambient noise as possible. We can 
rewrite (11) by assuming ||b|| 2 = 1 (i.e., scaling irrelevant) 

P (N, r, a, b) - [(ff2/ + | T ; +/22)b)b j (12) 

where Ri and R 2 are, as usual, positive semidefinite channel response 
autocorrelation matrices. Here Ri corresponds to precursor distortion 
while Ri relates to postcursor ISI. We notice that, if we form for 
^ a ^ 1 

Pa (N, r, a, b) = [(ff2/ + $+ aR2)h) b] , (13) 

we can continuously vary p a (N, t, a, b) from the decision feedback 
formulation where a = to the linear equalization case where a = 1. 
Thus, although we implement decision feedback equalization, it is 
possible to design the transmitting and receiving filters so that the 
amount of postcursor distortion is still mildly to strongly influential. 
Of course, a more general formulation of this "hybrid" design tech- 
nique is possible by retracing our steps back to (11) and forming 

p{N > T ' a ' b) = ,«+£ (Aj* T ,'b>+EU fcgt ,b7 2 ' (U) 
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where A k are (N + 1) X (N + 1) diagonal matrices (obviously 
Ak = I for the linear equalizer case). + 

2.2 Joint optimization of transmitter and filter design (baseband) 

In the individual design of transmitter and receiver filters treated 
in the last section, we were able to find the optimal filters by a simple 
rearrangement of interference terms and applying the Cauchy- 
Schwartz inequality. We find that for joint filter optimization this 
procedure will be slightly modified and additional steps will be taken 
to arrive at the solution. 

We recall that the total response of the system depicted in Fig. 1 is 

{r„} M+2Ar = {a n }$*lhnW*{b n }Z (15) 

and the signal-to-noise ratio : 

^'■•' b) %wW ! ' (16) 

k< T 

where c is the 2N + 1 dimensional vector formed from the sequence 
{a n }£*{b n }o and h fc = (h k , h t -i, • • •, hh-m). Here again, p(N, t, a, b) 
is seen to be a continuous function of a and b and functionally in- 
variant to the norm of b. Hence, we constrain our search for the 
optimal b vector by imposing ||b|| = 1. 

The transmitter power constraint was imposed in Section 2.1 by 
||a|| = m 2 - In practical situations, the constraint is more likely to be 
|| a || ^ p 1 . That is, we want to use only enough power to yield a suffi- 
ciently high signal-to-noise ratio at the receiver. For example, we 
constrain the receiver filter to be of unit norm since the norm is not 
going to contribute toward the enhancement of the signal-to-noise 
ratio at its output. Rather, it will be the transmitter filter power 
output which determines the output signal-to-noise ratio to a large 
extent. A way of solving the joint filter optimization problem with 
constraints, then, is by permitting the transmitter power level to be 
at that as-yet undetermined level so that the signal power through the 
transmitter, channel, and receiver will be at a prespecified ratio to 
that of the ambient noise. Hence, we have the following optimization 
problem : 

,*r kx (a*b, h T )* (m 

max p(N, t, a, b) = max , , ^ ,. . % 2 • K L ') 
llh*a*bii=„ ' iih*a*bn =„ <t 2 + Z (a*b, hfc) 2 

l|b||=l t|b||=l *<' 



f Of course, some constraint must be put on A* to make the maximization of p 
meaningful. 
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Proceeding as before, we obtain 

(a*b)* = fc ( - 2 H T H + fiY\ , (18) 

(where k is determined from the constraint ||h*a*b|| = 7;) with 

P (N, r, a*, b*) = (h T , {Ah*R + r)\) , (19) 

where H is the 2M + 1 X 2JV + 1 matrix such that H (a *b) = h*a*b 
where h = (h , hi, hz, • • • , h M ) ■ The matrix R is formed from the 
£ hk-xhk-j terms. Now (18) can be written in its z _1 transfer function 

fc<T 

representation. 

(a*b)*(z->) = fc(l + ax*- 1 + a 22 - 2 + • • • + a 2N Z~™) 

A {z- l )B{z-i) = fcQ 1 (0- 1 )Q 2 (z- 1 ) • • -Qrtir*), {ZV) 

where A; is a determined constant and the Q's are quadratic factors 
with real coefficients. A choice of the quadratic factors for composing 
A(z~ l ) and B(z~ 1 ) exists. However, since ||b*|| = 1 we are then left with 
a determinable norm for a*. For example, we might choose 

B*(z) = „% l( /\L (21) 



Hence, 
with norm 



\\Qi{**W 
A*(z~>) = \\Qi(z-')\\'kQ 2 (z-i)- • -Qniz- 1 ), (22) 

\\A*(z-i)\\ = *||Qi(r^)||||g,(«-»).»Q.(r-»)||. 

Regardless of how the quadratic factors are assigned, B*(z~ 1 ) is 
normalized and A*(z~ l ) is then left with some norm value which may 
be large or small. The total norm ||a*h*b||, however, was chosen to be 
7] and for each receiver filter chosen from the quadratic factors of (20), 
a corresponding ||A*(z _1 )ll results. It is of definite engineering interest 
to seek that quadratic factor combination which minimizes ||A*(z _1 )||, 
but no obvious solution exists for this combinatorial problem. Other 
considerations may come into play at this point which would obviate 
the need for minimizing ||A*(z -1 )||. For example, a minimum phase 
requirement for one of the two filters would delineate the two filters. 
Roundoff noise considerations for digital filter implementations might 
also contribute toward selecting one quadratic factor over another at 
the receiver. Cost considerations may warrant the splitting of the 
two filters into equal lengths (N even) so that the number of possible 
quadratic combinations is reduced considerably. In any case, this 
filter-splitting problem is akin to the quadratic factor placement 
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problem in minimizing roundoff noise in digital filter implementations. 
In Appendix D we outline a technique for separating the transmitter 
and receiver filters. 

Of course, it is possible to go through the same generalization on 
postcursor and precursor equalization that we did for the integral 
optimization problems of Section II. We obtain in that case 



where 



P {N, r, a*, b*) = \h T , ( jH*H + fli + aR^j h, 1 , 



(23) 



a*b = fc ( -1 H T H + Rt + aR>\ h T (24) 

and where #1(^2) is the matrix corresponding to precursor (post- 
cursor) interference terms and 2=» a 2a 1. 

III. PASSBAND FORMULATION 

It is possible to extend the results outlined in the previous sections 
to the passband equivalents of transmitter, channel, and receiver for 
a quadrature amplitude modulation (QAM) system. f The extension 
of results is not without complications, since QAM systems suffer 
from another form of distortion — co-channel interference (CCI). 
Thus, the transmitting and receiving filters will be expected to combat 
not only ambient noise and ISI but also co-channel intersymbol 
interference (CCISI). 

3.1 Integral optimization 

We begin by referring to Fig. 2 which illustrates the QAM system 
with decision feedback. We are interested in the transmitter and 
receiver filter designs so that a measure of transmission performance 
is maximized. Namely, we seek to maximize a sampled signal-to- 
generalized-noise ratio similar to that defined in (2). To define the 
terms which will appear in our performance measure, we note that 
the "in-phase" response at the receiver is 

- {aP}Z*Z[hP)¥*[bPW + l*P)¥W\fl (25) 

while the "quadrature" response at the receiver is 

+ {an^uhn^ww - \hpw*[bp\(r\. (26) 

+ We will not concern ourselves with the problems of carrier acquisition and timing 
for the QAM system we consider here in discrete form (see Appendix A for a discus- 
sion of these items). 
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Fig. 2 — QAM data system with decision feedback equalization. 

Each channel response has two ISI components, an in-channel inter- 
symbol interference term and the other due to co-channel interference. 
We notice that the CCI is completely eliminated if {H p) \ = [hP\ 
and \bi Q) \ = \hlP). However, in our considerations we will always 
assume M » N so that our filters do not have a sufficient number of 
degrees of freedom to eliminate CCI (also, this action does constitute 
suboptimal filtering). 

We form the in-phase resultant signal-to-noise ratio for independent 
input channels, independent symbols of unit magnitude with equal 
chance of occurrence and uncorrelated noise samples of variance a 2 . 
We first treat the case where the receiver filter is all pass (i.e., b = e, 
the identity vector in the algebra of convolution) 



P .(N, r, a, b) = 

b=e 



Now we define 



C(»w,h«) - (a ( *\ h< 9) )] 2 



<r 2 + Z [(a ( '",r4 p) ) - (a<*>, hn] 2 + 



fc<T 



+ Z [(a ( *\ hj») + (a"', hl«)J. (27) 



a = [a^>,a ( «>] 
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and h T = [h< p> , - h< fl) ] and h? = [h< 8) , h< p) ], vectors of 2JV + 2 unit 
length. Hence, we can rewrite (27) into 

p. (AT, T> a, b) = <r! + 2:(a ,h^' + i:(a,hf)' ■ (28) 

k<T k^T 

In (27) and (28) we have tacitly assumed that at the receiver each 
channel will "talk" to the other for the purpose of cancelling post- 
cursor CCISI also.'* That is, we have assumed a dual system of decision 
feedback equalization is being implemented. 

The maximization of p,(iV, t, a, b) subject to b = e and ||a|| 2 = m 2 
leads to a solution similar to that of (6) and (7) : 

# _ MDrVJ + Q + gjjt; 
a ||[>-VI + Q + QdrM 



(29) 



(30) 



max p(N, t, a, b) = p(N, t, a*, e) 

|tJt-M' 

n(h T , (m-VI + Q + Qc)-^,) 

lUy-vz + Q + QJ^M 

where Q and Q e are respectively the in-phase and co-channel correlation 
matrices similarly formed, as was the Q matrix of (6). The a* vector 
of (29) separates into a* (p) and a* (9) and the conditionally optimal 
transmitter bandpass filter is completely specified. Following the 
procedure in Section II, we now hold the transmitter design fixed 
at a* and rewrite (25) as 

- {bnz*{ia t ™}z*{hn?\ 

-lbP}?*llal<*}g*lhP\?} (31) 

= {bnZ*{{g? p) }? +N - teW 1 "") 

- {bnZ*{lgr}? +N + {gl pa) W +N }, (32) 

where { gi u) } q +n , u = pp, pq, qp, qq are recognizable from (31). 
We can now write the expression for p (N, t, a*, b) as 

(N , M __ [(b w ,g^')- (b« >g y)T 

p{JS, r, a , 0) - ff2|)b||2 + g [(b(p)j g ( P)) _ (b(9)j g («))]2 + . . . 

<T + Z [(b'o, gi p) ) + (b<», gTO 2 , (33) 



f Also, we are assuming we will eliminate all postcursor ISI. However, in practice, 
only a few postcursors would be removed. Thus, some postcursor terms would appear 
in the denominator of (28) in that case. 

512 THE BELL SYSTEM TECHNICAL JOURNAL, MARCH 1974 



where 

t — KUt Qt , Ut-i Qr-i, '",y r —N Qt-N), \o<k) 

and similarly for g' a) . To maximize p(N, t, a*, b) subject to ||b|| = 1, 
we first form the concatenated vectors of 2iV + 2 length : 

b - (b',b»), g* = (gi», -gi% gl = (gP,g?>). (35) 

Hence 

" (JV ' T ' a *' b) = ^iw + E(b b, g^+i:(b,g,r ■ (36) 

k<r fcgr 

and we proceed to find that 

max P (N, t, a*, b) =[g T , (<r 2 / + R + B^g,] (37) 

l|b||-l 



achieved at 



(<r*7 + ft + fio)- 1 ^ ,_, 

b " ||(o«J + B + i2 e )-ig T ||' (db) 



where R and .ft c are channel response correlation matrices of the type 
encountered before. 

3.2 Joint optimization 

To jointly optimize the transmitter and receiver passband filters, we 
follow virtually the same procedure found successful for the baseband 
case. A comparable factorization problem arises here, for which only a 
combinatorial solution seems to exist. 

The in-phase and quadrature responses through a passband trans- 
mitter, channel, and receiver are given by 

- [hl*}if*(laPW*\bPW + WSWISO (39) 

\ri*}?+* N = [h2»\?(\a<»)?*{b«>}? + {a<«>}MftH$0 

+ lhP\?*{[aP}ff*{bP}8 - WI^WID. (40) 

Rewriting (39) and (40) in terms of a combined passband filter with 
responses c<*> - \d p) }l N and c ( «' - [cFffi*: 

\ri p) }^ +2N = (*^|Wrf»}r - W}fr{cP\V (41) 

(rP) ?+*"= lA^IMcfir + {M^IWePjr, (42) 

we form the augmented vectors c = [c <?) , c w ] 
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and 
h? = (HP, hf»i, ■■■, tt«l 2N , h?\ • ■ ■ , h?i 2N ), hi n) = 0, A; < 0, n = p,q. 

Our signal-to-noise ratio becomes for the in-phase channel : 

(c h ) 2 
p{N > T > C) = <r 2 ||b|| 2 +(flc,cW(^c,c)' (43) 

where R and R c are the now-familiar channel correlation matrices and 
b = (b (p) , b (fl) )- It is easy to show that the norm of the receiver filter 
is irrelevant in the maximization of p(N, t, c). Hence, we choose 
||b|| = 1. We now specify the amount of signal power jj 2 we will need 
at the receiver upon choosing the optimal filters. That is, 

M+2N 

Z |(h fc ,c)| 2 +|(h?,c)| 2 = f. (44) 

*=0 



But (44) can be rewritten 



%«) = i, (45) 



where Q is a sum of two correlation matrices. Hence, (43) then yields 

the problem : 

(c,h T ) 2 



max 

(Qc,< 



(46) 



Sr=V^f^+(«c,c) + (« c c,c) 

to which the solution is 

c* = k(^ + R + BcYhr (47) 



and 



p(N,t,c*) = [h T ,(^ + R + R<) V]. 



The constant k is determined from the constraint that (Qc, c) = tj 2 . 
Since c = (c (9) , c (p) ) and the vectors (a (p) , a (ff> ) and (b«, b ( «>) all 
make up c (p) and c (9) , we encounter a factorization problem. We can 
choose (b (p) , b (fl) )» normalize the receiver filter, and then are left with 
the transmitter filter which has a given norm. This norm is then the 
transmitter power required to produce ti 2 /<t 2 generalized signal-to- 
noise power at the receiver. 

IV. EXAMPLES 

To illustrate the difference in performance between decision feedback 
and linear equalization, we have taken a telephone DDD toll connec- 
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tion as a linear channel model. Specifically, we would like to know the 
difference in performance on a telephone connection when both linear 
and decision feedback equalization schemes are constrained to use a 
finite number of taps. For comparison, we compute the performance 
asymptotes (infinite number of taps) for each equalization scheme 
(see Appendix C) realized when an infinite number of taps are available. 
We ask whether it is possible to approach these asymptotes with a 
reasonable (implementable) number of taps. Another point which is 
raised in every implementation of decision feedback equalization is 
that of postcursor size. If a mistake in symbol identification is made, 
then the subtraction, for example, of an erroneously signed postcursor 
may lead to a burst of errors if the postcursor size is large. We illustrate 
the postcursor sizes for a passband decision feedback equalization 
system operating on an average telephone connection. 

Figure 3 illustrates the magnitude characteristic of the average 
DDD toll telephone connection as measured in the 1969-70 Toll 
Connection Survey of the Bell System. The corresponding delay 
characteristic follows a parabolic shape and has been numerically 
integrated to yield a phase curve. As discussed in Appendix B, the 
bandpass channel parameters have been calculated for various carriers 
and various flat Nyquist spectral widths assumed at the transmitter. 
The spectral width was controlled by superimposing a cosine rolloff 
(400-Hz width centered at the Nyquist frequency) on the in-phase 
and quadrature spectra. Figures 4 and 5 show typical passband spectra 
computed for this channel. When this decomposition of the bandpass 
channel into in-phase and quadrature responses is achieved, it is 
possible to compute the performance asymptotes for linear and decision 
feedback equalization given in Appendix C. The result of these compu- 
tations is shown in Table I. It is seen that performance decreases 
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Fig. 3 — Average amplitude characteristic for toll telephone connection (from 
1969-70 Toll Connection Survey data). 
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Fig. 4 — Quadrature amplitude characteristic for average toll telephone connection. 

with data rate and slightly with increased carrier frequency. The gap 
between decision feedback and linear equalization widens as speed 
is increased. For all computations, we have kept the total transmitted 
power through the channel fixed at - 12 dBm, whereas the noise power 
spectral density was kept at that level corresponding to total noise 
power of -48.3 dBm over a 0-3000 Hz bandwidth. This noise level 
is 3 dB weaker* than the average noise power measured in the 1969-70 
Toll Connection Survey. 

A finite length receiving filter was increased in length until perform- 
ance was reasonably close to the asymptote given in Table I for that 
speed and carrier. Figure 6 illustrates the difference in performance 
between decision feedback and linear equalization. It is seen that less 
than half the number of taps are required by the decision feedback 
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Fig. 5— In-phase amplitude characteristic for average toll telephone connection. 



* Noise level was made weaker only for computational convenience. 
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Table I — QAM transmission asymptotic SNR in dB for average 

toll telephone connection 

Noise at receiver — 48.3 dBm; transmitted power — 12 dBm 





Carrier = 1650 Hz 


Carrier = 1700 Hz 


Carrier = 1800 Hz 


Rate in 








Baud/ch. 


L.E. 


D.F. 


M.F. 


L.E. 


D.F. 


M.F. 


L.E. 


D.F. 


M.F. 


2400 


31.4 


31.9 


32.1 


30.7 


31.6 


31.9 


29.2 


30.9 


31.6 


2600 


31.3 


31.5 


31.7 


31.1 


31.3 


31.5 


30.5 


30.9 


31.2 


2800 


27.3 


30.4 


31.1 


28.5 


30.2 


30.9 


28.8 


29.9 


30.5 


3000 


27.8 


29.8 


30.6 


27.8 


29.6 


30.4 


25.5 


28.7 


29.9 


3200 


25.5 


29.0 


30.1 


25.1 


28.6 


29.9 


19.4 


27.4 


29.4 



L.E. = Linear equalization asymptote. 
M.F. = Matched filter bound. 



D.F. = Decision feedback asymptote. 



equalizer to achieve a level of performance close to the asymptote. 
In addition, the linear equalizer even with its 36 tap length per channel 
could not keep an acceptable performance level when the data speed 
was increased to 3200 symbols/s/channel. On this basis, the premise 
that decision feedback equalization has significant advantages over 
linear equalization may be too readily accepted. For, if we examine 
postcursor sizes on one of these equalized bandpass channels, we can 
see that the high signal-to-noise ratio offered by decision feedback 
does not come without penalty. Figure 7 illustrates sample sizes of a 
toll telephone channel equalized with a 16-tap (8-fccdback) decision 
feedback equalizer. The precursors, or samples before the main sample 
peak, are too small to be seen on this scale. However, it is clear that 
the postcursor adjacent to the signal sample, which is greater than 
half the latter's size, presents a problem. Should a decision error occur, 
the next signal sample could have its polarity reversed, since more 
than twice its strength could be subtracted out by the decision feed- 
back processor. Thus, error propagation is possible with only a single 
mistake providing the ignition. Let us recall the hybrid equalization 
scheme discussed in Section II. We note in Fig. 8 that, for an alpha 
value of 0.01, we diminish the size of the large postcursor and more 
evenly distribute the heights of all the postcursors to be subtracted 
by the decision feedback processor. It is now apparent that no one 
postcursor is large enough to reverse the polarity of the signal should 
a decision error occur. It will take several consecutive decision errors, 
for example, before this can happen now. However, we lose 1 dB in 
signal-to-noise ratio for this example when we opt for this mitigation 
of the postcursor size problem. Of course, a trade-off exists between 
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NOISE AT RECEIVER = -48.3 dBm 
TRANSMITTED POWER = -12 dBm 
CARRIER FREQUENCY = 1650 Hz 

•8 TAPS ARE FEEDBACK 

Fig. 6 — Performance of finite equalizers for average toll telephone connection. 

the loss in signal-to-noise ratio and reduction of postcursor size by 
means of this method. 

V. SUMMARY 

We have treated the design of finite length transmitting and 
receiving filters for a data system employing decision feedback 
equalization. Our purpose here was to examine the difference in 
performance between linear and decision feedback equalization on 
a given data channel. Sequential and joint optimization of trans- 
mitting and receiving filters were treated for an all-Nyquist equivalent 
data system. Although the solutions for the optimum tap settings 
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Fig. 7 — Postcursor size problem and mitigation. 



and signal-to-noisc ratio were derived in general terms, applying the 
results to the spectrum of a toll telephone connection was of special 
interest. For this channel example, it was found that fewer filter taps 
were required for decision feedback equalization to achieve a reason- 
able performance level. The problem of postcursor size for an overall 
response of a passband decision feedback equalized system can be 
mitigated by a hybrid equalization scheme. The price for allowing the 
linear filter taps to diminish the postcursor sizes in this hybrid equalizer 
is a lower signal-to-noise ratio. 

APPENDIX A 

Details about the discrete channel model 

The lowpass filters in the A/D or D/A conversion process shown in 
Figs. 1 and 2 delimit the channel frequency band which supports data 
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Fig. 8 — Total impulse response hybrid equalization. 
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transmission. Hence, the channel can be seen as a bandlimited medium 
and can also be reduced to discrete form for M sufficiently large : 

n=0 Z 

where T is the data symbol interval or 1/2 T is the Nyquist frequency. 
The point here is that \h n ) is dependent on the timing chosen for 
this reduction. Obviously, a timing exists which maximizes a signal- 
to-noise ratio, for example, of the unequalized response. We have 
found by experimentation that this timing was an excellent approxi- 
mation to the timing which leads to a maximum signal-to-noise ratio 
after equalization. 

For a bandpass channel, the decomposition into discrete form takes 
place in two steps. First, a carrier frequency is chosen, and in- phase 
and quadrature spectra are then computed. A constant carrier phase 
is then a variable parameter. However, it is easily shown that this 
carrier constant can be absorbed by either the demodulation process 
or the passband equalizer tap settings. 

It is important to recall that the time samples of the spectrum 

M 

H(f) = £ h„ e-i» 2 " T 

n=0 

are {™-Mo f - Hence, in the formation of the signal-to-noise ratio: 

K 
p <r*T 2 + ISI 

we form the generalized variance parameter a 2 T 2 where <r 2 is the noise 
sample variance. This accounts for this transformation from Fourier 
coefficients \h n }^ to time samples {l/T-h n \". 

APPENDIX B 

Channel data from 1969-70 toll connection survey 

The average loss and delay measurements of over 600 toll voice- 
grade connections made in a 1969-70 survey are recorded in Ref. 4. 
For our channel model, interpolative curves were constructed from 
the average survey measurements made on 20 frequencies. A linear 
loss slope was appended at the lower frequency end to extrapolate loss 
down to zero frequency. The slope of the loss curve in decibels at the 
lowest measurement frequency (200 Hz) was used for this extrapola- 
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where 



tion. A constant was added to the integrated delay curve to achieve 
zero phase at zero frequency. 

Passband responses at several carrier frequencies were then formu- 
lated from the interpolated baseband data. An impulse response was 
calculated for each in-phase and quadrature channel. Timing for the 
two channels was chosen to maximize the squared sampled signal to 
mean square ISI and CCISI before the receiver filter. One hundred 
eight Nyquist samples ({/4 p) U=o and \hjp}h n ) represented each 
passband channel. 

APPENDIX C 

Asymptotic MSE as derived by Falconer-Foschini - 

We list here the formulas for the MSE as achieved by linear equali- 
zation and decision feedback for passband systems (here, independent 
binary ± 1 transmission is assumed with No/2 input noise spectrum) . 

(MMSE) linear = f™* r T( ^£P + l)~df (48) 

(MMSE) d i = exp j T f'"^ log ( ^- + l)"\lf J , (49) 

*•</) - ^ Z |Gi(/ + |) + ;»■(/ + |) f 

X | Cl ( / + f) +iC2 ( /+ l)| 2 " 

The passband transmitter and channel characteristics are denoted 
by (?i 4- jG% and Ci + jC 2 , respectively. For comparison purposes, 
it is simple to show that the matched filter bound is 

(mmse)„„ = |r JZX^r + 1 ) df }~ 1 - (50) 

It is of interest to note that we can prove that expressions (48), (49), 
and (50) follow the sequence 

(48) ^ (49) ^ (50) 

by invoking Jensen's Inequality for the logarithm as the concave 
function. It is clear that, for the ideal channel and transmitter, i.e., 
X (f) = l/T, we have 

(MMSE)u„ear = (MMSE)„, - (MMSE) mf = t + (y.r)-i - 

DFE FILTERS 521 



APPENDIX D 

A technique tor separating transmitter and receiver filters 

We wish to determine that factorization of 

A(r*)B(r») = nQ,(r'), 

n-0 

which minimizes HAfc- 1 )!! while ||5(ar x )|| = 1. Each 

Q n (z~ l ) = 1 + erf*r* + (4 n) z- 2 

is a quadratic factor. We assume no real roots occur, although the 
extension of the technique we present here to include real roots is 
obvious. Now 

||Q n ( 2 -i)|| 2 = p \Qn(e-fi« T )\ 2 df = 1 + (a[ n) Y + (aj*»)». 
We notice that, upon choosing B(z~ l ) = U Q«*(« -1 ) (where 

ntGATfl 

N B \JN A = {0,1,2, •••,#}), then 

\\A(z- l )\\ = || n Qn„{z- l )\\ II II Q-C*" 1 )!!. (51) 

nkGNB n- GN a 

Thus, what we really want to do is select a partition of the Q k factors 
so that the product of the norms of the partition factors is minimized. 
Much like the quadratic factor partitioning problem in digital filter 
implementation for minimizing roundoff noise, the only method for 
obtaining the global minimum of ||A(z _1 )|| seems to be the formation 
of all possible combinations of quadratic factors. When JV is large, 
say, 20, this combinatorial method is time-consuming even when the 
filters are forced to be of the same order. 

A technique for constructing the partition which sequentially 
minimizes \\A (z -1 )!! is first begun by reordering the quadratic factors by 
norm {Q n i)!L . We think of the two norms of (51) as bins, and we 
sequentially fill those bins with quadratic factors. We insert one of 
two quadratic factors of largest norms into the first bin and the second 
factor into that same bin. We evaluate the norm of the first bin and 
now compare it to the product of the norms of the individual factors. 
Whichever placement results in smaller norm product, we choose as 
our partition initialization. Thus, at the end of the first step we have 
either 

bin 1 bin 2 

IIQ-iQ-.ll Hill 
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or 

bin 1 bin 2 

IIQ.,11 \\Q»,\\, 

depending on which product is smaller. The next factor, Q„ t , is brought 
into the current partition and the products again are tested as to 
whether Q„ 3 minimizes the product when placed in bin 1 or bin 2. 
The process continues until all quadratic factors are placed into 
either bin. 

This procedure has been programmed and tested on actual filter 
quadratic factors. It has been our experience that the resulting factori- 
zation was close to the optimal one. To cite an example: Ten quadratic 
factors were randomly placed into two bins 500 times. The product 
of the norms of the two bins' contents ranged from 0.584 to 1183.33. 
The partition which our procedure yields for this set of quadratic 
factors had the product value of 0.646. Only 36 of the 500 partitions 
yielded smaller products. But little could be gained by using any of 
these 36 partitions. However, the worst partition was four orders of 
magnitude away from the outcome of our procedure. This is possibly 
what is most important, namely finding a partition very far away 
from the worst one. 
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